Humans and dogs

The MNIST dataset is a fun way to get your feet wet, but real images can require a bit more preprocessing to ensure that they are formatted correctly for input into keras. Let’s try another example with some folders of images!

The data

Load a folder of human and dog images from the data-raw folder of the workshop materials. First, we want to define the file paths of our already split training and validation (aka test data) image folders. These free images were downloaded from Burst.

# Specify the file paths for our training and validation (aka test) data
train_path = "data-raw/dog-person/TRAIN"
val_path = "data-raw/dog-person/VAL"

# Define two more variables as the actual names of the files
train_images = list.files(train_path, full.names = T, recursive = T)
val_images = list.files(val_path, full.names = T, recursive = T)

# We have 34 training set images and 16 validation set images
length(train_images)
## [1] 34
length(val_images)
## [1] 16
# Plot the first training image, and the first validation image
library(cowplot)
## Loading required package: ggplot2
## 
## Attaching package: 'cowplot'
## The following object is masked from 'package:ggplot2':
## 
##     ggsave
ggdraw() + draw_image(train_images[1])

ggdraw() + draw_image(val_images[1])

Define model characteristics

Now, we want to define some of the characteristics outside of our model. This way, we can just pass in the same variable instead of a value each time.

# Define image width and height as 96 pixels
img_width = img_height = 256L

# Define our batch size, or number of training observations for each epoch, as 5
batch_size = 5L

# Define the number of images in our training dataset to be used (34)
(num_train_samples = length(list.files(train_path, recursive = T)))
## [1] 34
# Define the number of images in our validation dataset to use (10)
num_validation_samples = 10L

# Define the number of full passes of the model 
epochs = 30L

With these setup variables defined we can now focus on actually building the model. image_data_generator performs a bunch of data augmentation. We will just use the rescale parameter to scale our data to what will essentially become a bunch of binary matrices - one for each image.

See ?image_data_generator to learn more.

# Transform our training data to rescaled values of 
train_datagen = keras::image_data_generator(rescale = 1/255)

# Transform our validation data
val_datagen = keras::image_data_generator(rescale = 1/255)

Remember that we can learn a lot more about our keras variables by using the dollar sign operator $. This functions in a similar way to dot notation in Python for calling methods. Press the tab key after the dollar sign in the cell below:

train_datagen$

Define the image characteristics

Before we define the model, we want to give keras a few more details about the properties of our images. We will use the flow_from_directory method on both train_datagen and val_datagen to define our new variables. What parameters are we specifying?

# Configure the training model
train_gen =
  train_datagen$flow_from_directory(train_path,
                                    target_size = c(img_width, img_height),
                                    batch_size = batch_size,
                                    class_mode = "binary",
                                    color_mode = "grayscale")

# Configure the validation model
val_gen =
  val_datagen$flow_from_directory(val_path,
                                  target_size = c(img_width, img_height),
                                  batch_size = batch_size,
                                  class_mode = "binary",
                                  color_mode = "grayscale")

Define the model

Define the model! Refer back to part1 if you need help remembering what the parameters are and what your arguments should be :)

library(dplyr) # %>% come from here
## 
## Attaching package: 'dplyr'
## The following objects are masked from 'package:stats':
## 
##     filter, lag
## The following objects are masked from 'package:base':
## 
##     intersect, setdiff, setequal, union
library(keras) # layer_flatten, layer_dense, layer_dropout
model <- keras::keras_model_sequential() 

model %>% 
  
  # INPUT LAYER
  # layer_flatten will turn our 3D array into a one-dimensional one
  # Note: we did not have to do this in part1 because the data were already flattened
  layer_flatten(input_shape = c(img_width, img_height, 1)) %>%
  
  # INPUT LAYER
  # layer_dense allows us to actually add the input layer. What parameters are we specifying?
  layer_dense(units = 20, activation = 'relu', input_shape = c(img_width, img_height)) %>% 
  
  # INPUT LAYER 
  # layer_dropout allows us to apply regularization to our model and can apply to all layers
  layer_dropout(rate = 0.1) %>% 
  
  # HIDDEN LAYER 
  layer_dense(units = 20, activation = 'relu') %>%
  
  layer_dropout(rate = 0.05) %>%
  
  # OUTPUT LAYER
  layer_dense(units = 1, activation = 'sigmoid')

summary(model)
## ___________________________________________________________________________
## Layer (type)                     Output Shape                  Param #     
## ===========================================================================
## flatten_1 (Flatten)              (None, 65536)                 0           
## ___________________________________________________________________________
## dense_1 (Dense)                  (None, 20)                    1310740     
## ___________________________________________________________________________
## dropout_1 (Dropout)              (None, 20)                    0           
## ___________________________________________________________________________
## dense_2 (Dense)                  (None, 20)                    420         
## ___________________________________________________________________________
## dropout_2 (Dropout)              (None, 20)                    0           
## ___________________________________________________________________________
## dense_3 (Dense)                  (None, 1)                     21          
## ===========================================================================
## Total params: 1,311,181
## Trainable params: 1,311,181
## Non-trainable params: 0
## ___________________________________________________________________________

Defining the output metrics

We can use the generic compile function to specify our loss and optimizer functions and our classification metrics.

model %>% compile(
  loss = 'binary_crossentropy',
  optimizer = optimizer_adam(),
  metrics = c('accuracy')
)

Train the model and evaluate its performance on new data

Note that we have to use fit_generator to fit our model this time because we are also using the custom flow_from_directory function rather than the simpler format of part1.

history = model %>%
  fit_generator(train_gen,
                steps_per_epoch = as.integer(num_train_samples / batch_size),
                epochs = epochs,
                validation_data = val_gen,
                validation_steps = as.integer(num_validation_samples / batch_size))

# Review fitting history.
plot(history) + theme_bw()

# model %>% evaluate()

Challenge 1

Why were our loss and accuracy metrics so jumpy? Is this model any good? What are the differences between training and validation datasets? What might be done to improve our predictions?

Challenge 2 (take home)

Recapitulate what you learned in parts 1 and 2 and fit a deep neural network on the FashionMNIST dataset. How are these data stored? Are they actual images? How can we read them into keras in R?